Accelerating multi-objective neural architecture search by random-weight evaluation

Abstract For the goal of automated design high-performance deep convolutional neural networks (CNNs), architecture search (NAS) methodology is becoming increasingly important for both academia and industries. Due to costly stochastic gradient descent training CNNs performance evaluation, most existing NAS methods are computationally expensive real-world deployments. To address this issue, we first introduce a new estimation metric, named random-weight evaluation (RWE) quantify quality in cost-efficient manner. Instead fully entire CNN, RWE only trains its last layer leaves remainders with randomly initialized weights, which results single network seconds. Second, complexity metric adopted multi-objective balance model size performance. Overall, our proposed method obtains set efficient models state-of-the-art two spaces. Then obtained on CIFAR-10 dataset transferred ImageNet validate practicality algorithm. Moreover, ablation studies NAS-Bench-301 datasets reveal effectiveness estimating compared methods.